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(57) Abstract 

A data packet switching 
and server load balancing device 
is provided by a general-purpose 
multiprocessor computer system. 
The general-purpose • multiprocessor 
computer system comprises a plurality 
of symmetrical processors coupled ; ^ 
together by a common data bus, a main v.o 
memory shared by the processors, and > . 
a plurality of network interfaces each 
adapted to be coupled to respective 
external networks for receiving and 
sending data packets via* a 1 particular 
communication protocol, such . ,as 
Transmission Control Protocol/Internet 
Protocol (TCP/IP). A first one of the 
processors is adapted to serve as a 
control processor and remaining, ones 
of the processors are adapted to serve . 
as data packet switching processors. 
The data packet switching processors 
are each coupled to at least one of the 
plurality of network interfaces. The 
control processor receives raw load 
status data from the external networks . 

and generates load distribution configuration data therefrom. The load distribution configuration data is stored in the main memory for 
access by the data packet . switching processors. The switching processors route received ones of the data packets to a selected one of the 
external networks in accordance with information included in a header portion of the data packets and the load distribution configuration 
data. The switching processors perform periodic polling of corresponding ones of the network interfaces to detect a received one of the 
data packets therein. In addition, the switching processors re- write the routing information included in the header portion of the data 
packets to reflect the selected one of the external networks. 
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COMPUTER DATA PACKET SWITCHING AND LOAD BALANCING 
SYSTEM USING A GENERAL-PURPOSE MULTIPROCESSOR 

ARCHITECTURE 

5 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates generally to computer data 
communication networks, and more particularly, to a general-purpose 
10 multiprocessor computer architecture adapted to transfer data packets 
between computer networks and to perform load balancing to ensure efficient 
utilization of the computer networks. 

2. Description of Related Art 

Computer networks are widely used as a way to communicate 
15 messages between computers. The Internet is made up of more than 
100,000 interconnected computer networks spread across over 100 countries, 
including commercial, academic and government networks. Originally 
developed for the military, the Internet has become widely used for academic 
and commercial research. Today, the Internet has become commercialized 
20 into a worldwide information highway, providing information on every subject 
known to humankind. Simiiarlyr businesses and other entities have adopted 
the Internet paradigm as a m6cfel fbr ItKeir^ triterh'ai ' networks, or so-called 
'"intranets." " v, - ! '" ■ ' •'^■'■'■^ -ro; ■■ .-^ v --^--, ,: ; . . ,,; 

Messages transferred -.between computers within , £ network are 
25 typically broken up into plural data packets. Packet switching systems are 
used to route the data packets to their required destination and enable the 
efficient handling of messages of different lengths and priorities. Since each 
data packet includes a destination address, all packets making up a single 
message do not have to travel the same path. Instead, the data packets can 
30 be dynamically routed over the interconnected networks as circuits become 
available or unavailable. The destination computer receives the data packets 
and reassembles them back into their proper sequence to reconstruct the 
transmitted message. 
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• Internet computer, networks generally use the TCP/IP communications 
protocol, i which is an: acronym; for -Transmission Control Protocol/Internet 
Protocol tThe TCP portion of .the protocol provides the transport function by 
breaking a. message into smaJler . packets,: reassembling the packets at the 
5 other endrof the communication network; and re-sending any packets that get 
lost along the way. The JP portion of the, protocol provides the routing function 
by giving the data packets an -address for the. destination network and client at 
the destination network. .: Each data, packet communicated using the TCP/IP 
protocol includes a header portion thatcCqntains the TCP and IP information. 

10 Another communication, protocol vused in.; communication between Internet 
computer , networks is UDP/IP,' in ; which , UDP is an acronym for User 
Datagram Protocol. UDP is used 'in/ place, of TCP in conditions when a 
reliable .delivery is not required. , For example, UDP/IP is often used for real- 
time audio and video traffic where, lost -data packets are simply ignored, 

15 because there is no time to retransmit. ,Since the computer networks 
connected to. the Internet may user other communication protocols besides 
TCP/IP or .UDP/IP, gateways- are used, to convert data packets from these 
protocols into the other protocols. - , 

At a destination network, one or more routers may be utilized to receive 

20 incoming data packets and . route the::packets to other internal networks such 
1 as locaJ area networks.. (LAN); c ^The internal networks may further include 
■ xservers that^supply Jnformationv to one^or more clients. The servers are 
o generally high-speed microcomputers, miniqomputers or even mainframes. In 
some cases, the clients are internal to the network (i.e., at the back-end), and 

25 . the router acts as a conduit for communication of -data packets between the 
clients and the outside world. The back-end servers may provide various 
application functions for the clients, such as a database server that maintains 
the databases and processes requests from; clients to extract data from or 
update the databases. In other. cases, the clients are external to the network 

30 (i.e., at the front-end), and the router acts as a conduit for communication of 
data packets between the clients and the back-end servers. For example, an 
Internet application server at the back-end may host Web applications within 
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the .network that are i accessed^ by vpUents^outsidfej the, network. In still other 
cases; the clients are both intemah and -external ta the networks The routers 
perform the functions of switching-; dala packets between i the internal and 
external networks, and balancing rthe load placed upon.the back-end servers 
of the internal network by distributing^message packets between the back-end 
servers in the most efficient and expeditious manner. . ' c : . - . • ■ 

In view of the high" volume of ^message traffic that they process and the 
relatively limited kinds of tasks that .they perform, routers typically comprise 
dedicated switching processors ' having an architecture optimized to provide 
these functions. A drawback^of such dedicated- switching. 1 processors is that 
they can be very expensive r dae in' part because they are manufactured in 
relatively low volumes as compared with other general-purpose computer 
systems. Moreover, the software) that provides ^the.messageJ'routing and load 
balancing functions must <be written specifically for. the dedicated switching 
processors, which further increases the cost of purchasing, operating and 
maintaining such systems:' An r additional drawback of dedicated switching 
processors is that most modifications: to their .functionality require a hardware 
change, which is typically more expensive and difficult than: a: software 
change. ■ « v • i. ..i v:* -yr* 

It would therefore b# ^ry desfrable:to:pravide the message routing and 
load balancing functions-> Wa network \-*teMerA within a ^ general-purpose 
symmetrical multiprocessor (SMP) cbmputerrsystem;.?- Such .general-purpose 
multiprocessor computer systems'' are n less ^expensive- rthahv conventional 
systems due to their larger ^volume production, Z^and t changes to • their 
functionality can be readiiy accomplished by modifying their software rather, 
than their hardware. - ; . 



' SUMMARY OF THE INVENTION 
In accordance with the teachings of the . present invention, a data 
packet switching and server load balancing device is provided by a general- 
purpose multiprocessor computer system: The general-purpose 
multiprocessor computer system comprises a plurality of symmetrical 
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processors coupled together by a ! commbrf. data: bus, a main memory shared 
' ' by the processors; arid 'a ''plurali^W : heWrork-. interfaces-each adapted to be 
coupled to respective external networks for receiving and sending data 
packets via a particular' communication : protocol, such as Transmission 
5 Control Protocol/internet Protocol (TCP/IP) 6r User Datagram Protocol (UDP). 

More particularly; a first one of the' processors is adapted to serve as a 
control processor and remaining ones of the processors are adapted to serve 
as data packet switching processors 5 ;" Trie* data packet switching processors 
are each coupled to at least one of the plurality of network interfaces. The 

10' control processor receives raw : load Tstatus data from agents running on the 
back-end application servers and generates load distribution configuration 
data therefromY The load distribution configuration data is stored in the main 
memory for access by the data packet switching processors. The switching 
processors route received ones of the data packets to a selected one of the 

15 ' external networks in accordance with information included in a header portion 
of the data packets and the load distribution configuration data. The switching 
processors perform periodic polling of corresponding ones of, the network 
interfaces to detect a received : one of the data packets therein. In addition, 
the switching processors' re-Write the-' routing information included in the 

20 header portion of the data packets to Reflect the selected, one of the external 
networks. "' "' " • • ■ ^ i^--: v . 

: . ^ m0 re"c6'm^ packet switching 

and load balancing system using a general purpose, multiprocessor 
architecture Will be" afforded to those skilled in the art, as well as a realization 
25 of additional advantages and objects thereof, : by a consideration of the 
following detailed description of the preferred embodiment. Reference will be 
made to the appended sheets of drawings, which will- first be described briefly. 

BRIEF DESCRIPTION OF THE DRAWINGS 
30 Fig. 1 is a block diagram of a network configuration having a load 

balancing and packet switching device in accordance with the present 
invention; 
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Fig. 2 : is . a block i diagram .. v pf a ger\eraJ-purpose. symmetrical 
multiprocessor computer: system . adapted >, to prpvjde. the. Jpjd, balancing a 
packet switching device;- v ;^r, s,,::^;,.^ ; - : c* 

Fig. 3 is '~a block .diagram;^ multiprocessor 
5 . computer system : configured ;.to ^provid? 9 .switching 5 prpcessor-tq perform 
■ network data packet switching^ sradira^cqntroj .processor, tp perform network 
load balancing; : V: = : - . ?ii 

Fig. 4 is a block diagram? depjcting communicatipn , of information 
between the control processon aqd^ne of the switching <proq?s,?ors; 
10 Fig. 5 is a flow chart: illustrating jopenatioa of, the, packet .engine module 

of the switching processor; c,:i : ,.■ 

Fig. 6 is a flow chart'.il^stratmg u pperatiQn of the. packet fij|er module of 

■ the switching processor; *v^i;r:v^ •' . . 

Fig. 7 is a block diagram illustfating a first embodiment of the invention 
15 having a pseudo-interface>i; between ^h^ switching 
processors through the; internal switch/; ^ r :f , :j > 

Fig. 8 is a block diagram 7 Hiustrating ; a second embodiment of the 
invention having a pseudo^tntgrfece and 
switching processors mroughjta^iyer;operati^g on.th^ pontroj processor; 
20 Fig. 9 is a block diagram jHustrating the pprtion^ of a data packet; 

Fig. 10 is a block diagram illustrating a third embodiment of the 
•invention having a user-levekfte^oi^ operating on 

the switching processor . / - * -ns. 4 ^ ; r ^ . v ; 

Fig. 11 is a flow chart 1 iljustrating process of initializing the switching 

25 processor for user-level access to the network interfaces; 

Figr12 is a flow chart illustrating a process of sending data packets to 
' a network interface at the. user level; and 

Fig. 13 is a flow chart illustrating a process of receiving data packets 
from a network interface at the user level. 
30 - • : ; 
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DETAILED DESCRIPTION' OF THE PREFERRED EMBODIMENT 
: r 'the present irWerrtibn ; satisfies" ,; tHe ; neeci for a general-purpose 
multiprocessor computer system adapted to provide message routing and 
loacl balancing fanctibhs v Yor a cbnipJfer network. In the detailed description 
5 that follows* like elerrieht nunier*atW ; are "used to describe like elements 
depicted in one or more of the figures. 

Referring first to Fig. 1, ah exemplary network configuration using a 
load balancing and packet switching system 10 of the present invention is 
illustrated. The network elements illustrated to' the' left of the load balancing 
10 " and packet switching system Id' in Fig. 1 " are referred to as the "back-end 
* server" side of the network, 1 and the : hetwbrk : elements illustrated to the right of 
the load balancing and' packet switching System .10 are referred to as the 
/'client" side of the network. On the- client side , of the network, the load 
balancing and packet switching system IP, is, coupled through two separate 
15 network channels to an external network switch 13. ;The external switch 13 is 
'coupled to client: stations Ja-1h permitting communication between the client 
- stations arid the back-end server side pf the network. The external switch 13 
is further connected to the internet ;(or : an intranet) 8 servicing client stations 
9^92 through a router 6. On the back-end server side of the network, the load 
20 balancing/ and packet switching .system I P: is, coupled through two separate 
< network channels . tbjari. .internal network switch 11 . The internal switch 11 is 
further coupled to to^ packets originated at 

~r the client side of the. network, such as- from- client .stations 5i-9 2 or 7v7 2) and 
directed to the back-end side of the network pass, through the external switch 
25 13 to the load balancing and packet switching^system 10, which determines 
the routing of the data packets to the back-end .servers 5^52-through the 
internal switch 11. Conversely, data packets, originated at the back-end side 
of the network and directed to the client side of the network follow the same 
path in reverse. : 
30 As known in the art, a network switch is a device that cross connects 

network nodes or LAN segments and allows full bandwidth to pass between 
connected nodes. Alternatively, the internal or external switches 11,13 could 
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be provided by, a network -hub, which; is. a >device that connects nodes by 
sharing the bandwidth, between: the connected node Network switches are 
advantageous over network hubs in.yiew.of their greater capacity and speed. 
.As also known in the art,. a v rputer is. a. device that route,s data jackets 
5 between networks. Routers., read the. network, address in each transmitted 
data packet and make a decision on how. to send it based on the most 
expedient route (traffic load, lir\e s cost$, speed, fc>adjjnes, etc.). Alternatively, 
the router 6 may be providgcjLby a network svyitch or hub. It should be 
appreciated that various, alternative ; network configurations are anticipated, 
10 and moreover, that the . numbers of clients, back-end servers and network , 
channels shown in Fig. 1 -are purely, for .the purpose, pf illustration and. are not 
intended to limit the scope of, the invention in any. way.. ^ . , t 

Referring now to Fig; 2, there is shown a general-purpose symmetrical 
multiprocessor (SMP) computer adapted to .provide the load balancing and " 

15 packet switching system TO 1 *of Fig. : -1: The SMP . computer : includes N f 
individual processors 24 0 -24 N coupled to : a common system f buis 12. Each one 
of the N processors 246-24 N has v ah associated cache memory 25 0 -25 N .: . The 
processors 24 0 -24 N may be provided toy^ 64Tbit UltraSPARC microprocessors " 
sold by Sun Microsystems^ Ihc- The SMP computer further includes, a main 

20 memory 14 and a memory controller 15 coupled to the xojTimoa system bus y 
1 2. The main memory 1 4* cohtaitis stored data and instructions accessible by 
each of the processors '24 0 -24|<ii witlT^lhesmernory: 6dntroHerq15 iControlling 
individual Accesses to -the main 'mefnory:^ As^nown^ in therart v :th;e cache 
memory 25b-25 N bridges the main memory 14 and the processors 24 0 -24 N . 

25 The cache memory 256-25^ is "faster than the main memory 14 and allows > 
instructions to be executed and data to be read at higher speed. Instructions 
and data are transferred to the cache memory 25 0 -25 N in blocks using a look- 
* ahead algorithm. The more sequential, the instructions in the routine being 
accessed, and the more sequential the order of the data being read, the more 

30 chance the next desired item will still be in the cache memory 25 0 -25 N , and 
the greater improvement in performance. It is , anticipated that the cache 
memory 25 0 -25 N be comprised of static random access memory (SRAM) 
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: , chjps )i; vyhile for main memory 14. 

Alternatively, the cache. .memory ( 23o-25n" may be .provided directly onto the 
same chips as the corresponding processors 24o-24 N - . 

An input/output' (I/O) controller; 16 is also coupled to the common 
5 system; bus 1 2^ and controls the- transfer of data between the processors 24 0 - 
24 N and peripheral devices, In particularr.the l/Q controller 16 is coupled to a 
disk interface device 18 : which controls exchanges of data between the 
processors 24o-24n and one or more disk-storage devices. The I/O controller 
16 is also. coupled to M network interface devices 17<i-17m which each control 

10 exchanges; of data between the processors 24o-24n and external computer 
networks, clients, or servers. Each one .Of thjSnnetwork interface devices 17r 
17 M include a receive queue m which received data packets are temporarily 
held while awaiting processing bythe SMP computer, and, a transmit queue in 
which . transmitted data, packets are temporarily held while awaiting 

15 communication ito a computer network.. It ■, should be appreciated that the N 
number of processors 24 0 t24n would generally be equal to or less than the M 
.number of, network interface . devices j17i-17 M . Each of the M network 
/interface devices 17r1 7m- may communicate with. plural computer networks, 
clients or? servers, using conventional netvyorkr protocols such as Ethernet, 

20 , ;Token Ring, Asynchronous Transfer Mode (ATM), etc. 

It should be appreciated that the SMP computer. may further include a 
.keyboard ,and • monitor -{not : shown) -to^ permit access- by management 
..information services, (MIS) personnel, such, as jtq, perform diagnostics, routine 
, maintenance, and administrative level tasks. As will be' further described 

25 below, the SMP computer is adapted to provide message routing and load 
balancing functions that would not require any direct user interaction, and the 
keyboard and monitor would therefore serve little use during ordinary 
operation of the computer system. However, certain applications of the load 
balancing and message routing system do, include user applications running 

30 on the SMP computer, and for. such applications it should be appreciated that * 
a keyboard and monitor would .be necessary. It is anticipated that the SMP 
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computer include a multitasking; muliiprocessfng'. operating system, such as 
the Solaris operating system.. by "Sun" Miorosysterns^lhcv - 1 - ' * 

Referring now to Fig. 3; a : block'.diagnam^of vthe gfcherail^purpose SMP 
computer configured to provide network - data \packet s\A/itching- and load 
5 balancing functions is illustrated. : ln ; the^oadibalaheing^ and packet switching 
system 10, one of the plural processors 24o-24m of Fig: 1 serves as a control 
processor 42, and the /emaintng : process6rs serve as switching processors 
44i and 442. The control processor 42 and switching processors 44! and 442 
each have access to a shared- memory space 34, such Sis ' provided by a 

10 portion of the main memoiy^14^^t Fig/ =t. The control processor 42 handles 
administrative and configuration Unctions for the^Io&d balancing and packet 
switching system 10, and also* communicates : with agents orr the application 
servers to collect system l0actHnfbrmation. The control - processor 42 -then 
performs complex calculations i on : the raw system load information and 

15 defines an optimum traffic load>distribution. . The traffic 'load distribution result 
is then written into the shared . memory .space for use by the switching 
processors 44! and 442. The switching -processors 44-j and 44^ exclusively 
perform the packet switching tasks, 3ndi do not handle any othbr computing 
tasks. Although two switching' proCessors''44i and : 442 are* depicted in Fig; 3, 

20 it should be appreciated thaf ^ny number of switching processors can be 
advantageously utilized.^ c HJd eAi u rf; ^£,o:*v"«<~ i-\ . ■:■ J -* 

The switching processoTs ^44^n , a , nd^44 , 2 i rgreoiea^h coupled -j-t© plural 
network interfaces 37i-37 3 , • such ? as-provicted "by (the' network* interfaces 1 7i- 
17m of Fig. 2. Each respective one of the switching processors 44V arid 44 2 

25 poll corresponding ones of the plural network interfaces 37-I-373 for incoming - - 
data packets 20 present on their respective receive queues. ' Particularly, 
: switching processor 44i polls the receive queue of network interface 37 t , and 
switching processor 442 polls the receive queue of network interfaces 37 2 and 
373: Since each of the switching processors 44^ and 44 2 poll different ones of 

30 the network interfaces 37 r 37 3 , conflicts between the switching processors 
over received data packets is avoided. In contrast, each one of the switching 
processors 44i and 44 2 can supply data packets to the transmit queues of 
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* each bnekof the network interfaces 37jV373i'so that data packets can then be 
routed to any computer- rieto the* load' balancing and packet 

switching system TO. ; « * - " : ^ ■ . 

Fig. -4' Illustrates in greater -detail -the communication of information 
5 between the control processor 42 and one of the switching processors 44. 
The control processor 42 further includes several software modules to handle 
discrete control tasks, including a resource: manager module 52 and a master 
module 54. The control- processor : 42- -may further include specialized 
application program interfaces " (API) that handle communication between 
10 these software modules. The resource manager module 52 receives raw data 
" from the back-end application servdrs^indicating their present load status. 
This raw data includes various'; factors; .including the number of clients 
presently being served, the utilization rates of the CPU and memory of the 
application server processor, the average execution time, and the number of 
15 ' requests per second. The raw load data us then provided to the master 
module 54; which synthesizes the" data into a desired load distribution in 
accordance'with a predetermined distribution .algorithm. For example, the 
distribution algorithm may favor distribution of incoming packets so that all 
• - servers have an even load, or alternatively: -may favor distribution of incoming 
20 packets to certain servers -having uriiqUe^applications or processing capability. 
Such distribution algorithms are 1 weH known in the art. It is also anticipated 
" J ' that the resource manager modi!Jl"e r 5'2«" , canc:be:provided as a separate device 
'entirely external to th : e ; cdntroJ : processor 42 : . 

The shared memory '34- further includes a routing table 62, a 
25 configuration table 64, and a connection table 66. The routing table 62 is a 
database that contains the current network topology, and is accessed by the 
switching processor 44 in determining routing information for the received 
data packets. Specifically, the routing table 62 defines the addresses and 
interconnection pathways between the load balancing and packet switching 
30 device 10 and the networks connected thereto. A routing daemon 58 within 
the control -processor 42 is a program that executes in the background to 
retrieve the information stored in the routing table 62 and maintains the status 
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of the routing table 62 as.. changes : are : :made to ...the configuration. ( As 
generally known in the ant, the: routing: daempn..58 functions like an extension 
to the operating system, and does not otherwise interact with the other 
j modules of the control processor: 42: or ttje. switching; prpcesspr 44. discussed 

5 above. . ; i * r ,:r- t - : v.- -<- 7 :rr. < ' j • / -•■ 

: The load distribution data- synthesized, by : the ■ master module 54 is 
stored in the configuration table \64.,r The configuration table> includes, two 
redundant memory buffers; identified; in Fig.. 4 as A and B. At any given time, 
one of the two memory buffers is the active buffer and the other is the back-up 

10 buffer. A memory pointer, withirn the shared memory 34 defines which one of 
the two buffers is currentl^ihe lacitiye : buffer. The . switching ; processqr 44 
obtains the current load :distributiorL data from the active buffer. The master 
module 54 of the control processor 42 periodically provides updated load 
. distribution data to the shared memory 34 thafe is.written to the, back-up buffer. 

15 Thereafter, the memory pointer! switches from the active to the back-up buffer, 
so that the updated load distribution - .data : is .accessible, to the switching 
processor 44. This double buffering technique speeds .up ..operation .of the 
load balancing and packet -switching . system. ; ;10 by enabling load ^data 
processing to occur concurrently.with . packet switching, ,and prevents potential 

20 . conflicts between the switching:: prpcesspr. 44 f and ^he control f processor 42 - 



that both need access to the; sa^eimemo^yispace; 



The connection-- table 66a.mafntatns M a : regret : of ..the TCP : and : ,UDP 
connections routed by each of ^the switching, proctors .44.,,^ 
above, the data packets received by the switching processors 44 each contain 

25 transport data in the header (Let, TCP or UDP data) that defines how the data 
packets should be reassembled with other data packets to reconstruct 
complete messages, or connections. As shown in Fig. 9, the data packets 20 
generally have an IP address which is provided in an IP header 20c to define 
the destination device as known to the external computer networks. This 

30 external IP address may actually be different than the internal IP address of 
the back-end application server selected by the load balancing and packet 
switching system 10. Accordingly, the entries of the connection table 66 map 
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the external IP address to the internal. IP address. Following the IP header 
^Oc, a TGP r (dr UDP) 1 header 20b contains, the transport data. The data 
portion 20a of <the data, packet 20 is provided after each of the foregoing 
headers. Returning now to-Fig: 4, a new entry, is added to the connection 
5 table 66 after a Tirst data packet, of a new connection is received. The 
transport data for each of the received , data packets is provided to the 
connection table 66 by the switching processor 44. 

Onceihe IP address is translated; by the connection table 66, the 
switching processor 44 determines 'a Media Access Control (MAC) address 
10 using an address resolution protocol (ARP). According to the ARP, a remote 
- network node desiring to transmit a data? packet to another node transmits an 
ARP broadcast packet that is received by every node connected to the 
network. The receiving node responds -with an ARP response packet that 
1 contains the MAC address of the receiving node. Thereafter, the remote 
15 network node uses the MAC address in a MAC header 20d of subsequent 
data packets. The remote network node then saves the MAC address in the 
ARP cache memory so that it won't, need to; send another ARP broadcast 
packet again/ • ' '< r- - ; ; 

Like the control processor, the switching processor 44 also includes 
20 software modules to'handle discrete tasks,;induding a packet engine module 
72 and ;1 a packet filter ^: module ,.;74.i;^;^The packet engine module 72 
communicates; with • the -network interface 37 to : periodically poll for the 
.' presence of data packets in the receive ; queue,; and delivers packets to the 
transmit queue to be sent to the. external networks: The packet filter module 
25 74 reads the IP and TCP/UDP data in the packet header to determine how to 
route the data packet. The packet filter module 74 accesses the connection 
table 66 in the shared memory 34 to determine whether a received, packet is 
part of an existing connection or a new connection. Then, the packet filter 
module 74 accesses the configuration table 64 to determine the proper 
30 -routing of the received data packet based on current load conditions and other 
factors. The switching processor 44 may further include specialized APIs that 
handle communication between these software modules. 
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The flow chart of Fig;:- 5 Illustrates the software- process performed by 
the packet engine module 72 iof t Fig. i4\2 Tfesoftw^re.^prpcess operates in a 
: tight loop so as to minimize the latency: of data, packets waitingjn the network 
interface receive queue. The process is initialized at* step, 100, during which 
5 : the switching processor 44 performs. csdainr. startup, tasks, including. checking 
the routing table 62 in the shared memory. 34. It is, anticipated that the load 
balancing and packet switchings/device? -10*. remain -continuously in an 
operational state, and so thisiinitiaHzation step may only be executed rarely. 
A first processing lodp begins with. step .1 02, in which the packet engine 

10 module 72 polls the network ; interface 37 receive queue. :: r At step 104, the^ 
packet engine module 72 : -determines whether there, are any data packets 
available at the receive- queue." If ho.data packets are available, ;the packet 
engine module 72 changes* to 1 the- next network interface 37 at step 106. As 
discussed above, a single switching: processor 44 may be responsible for 

1 5 receiving incoming data packets from plural ones of the network interfaces. It 
should be appreciated, however, -that: if the -switching .processor 44 only has 
responsibility for one network interface 37, then, this, step; 106 may be 
bypassed. After step 106, the packet engine module 72 returns, to step 102. 
This first processing loop. will: repeat indefinitely -until a received, data packet is 

20 detected at step 104. ; lf 'adata^paoket ^:avgilablevin^he : -network interface 
receive queue, a second processing da&p; begins al : s{ep : 1 08 at. which the 
packet engine module 72 retrieves; ^thendata paj^etnyiThen^a.t^step.l 10, the 
retrieved data packet is passed to the- packet Jltet module 74 for routing 
(described below); Thereafter, at step 1 12, the packet: engine module 72 

25 determines whether additional jackets are present at the network .interface 
receive queue.'* If additiohat packets are present, the packet engine module 
72 returns to step '108 and the second processing loop is repeated. If no 
additional packets are present, the packet engine module 72 returns to step 
1 06 and the next network interface is polled. 

30 The flow chart of Fig. 6 illustrates the. software process performed by 

the packet filter module 74 of Fig. 4. The process is initialized at step 200, 
during which the switching processor 44 performs certain start-up tasks as in 
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• step 100 discussed above: At step .2Q2y.the packet -filter module 74 begins 
processing of a data packet .retrieved"; by: the-packet engine module 72 as 
discussed above ; . The packetfilter module, 74 reads- the TCP/IP or UDP data 
from the header of ; .the data packet ia.step . 204. : The TCP/IP or UDP data will 
5 determine the subsequent processing, and routing of the data packet. At step 
; 2G6, the packet filter module 74 ^determines from the TCP/IP or UDP data 
whether the data packet is a valid service, entry. . In other words, the packet 
filter module "74 verifies that the^data. packet: was: properly routed to the load 
balancing arid packet switching device \1\0;.or..whether it was routed improperly 
10 and received by> the network interface in- error. If the data packet is not a 
valid service entry, at step 208,: the packet filter.module 74 sends a TCP reset 
packet back to ; the originator of ar TCP .connection via. the packet engine 
module 72 arid the network interfaces* on simply, discards the data packet of a 
UDP connection; «' 
15 • Assuming that the data packet is :aivalid service entry, the packet filter 

module 74 determines -at' step 210 s whether the data packet is a new 
connection' with a client. The packet-fitie^module 74 checks the transport 
data-in the data packet header against the entries in the connection table 66 
in. the shared memory 34 to determine - whether previous data packets have 
20 :, faeert received' from -the'-sartevdiefit.0reiV«>Usl|(,2dlfft- is a new connection, then 
=' the packef-filtef rfiddufe -74 ' eh.e^ table 64 for the current 

'■' -'load; conditio^ packet. As discussed above, 

' the 'packet filter- rtrodule ; 74: - may i elect-- ;to send . the data packet to the 
application server having the lightest current load. Alternatively, the packet 
25 filter module 74 may send the data packetito.a certain one of the application 
servers based on particular aspects of the data packet,, e.g., the data packet 
is part of a connection requiring processing capability unique to one of the 
application servers, or the data packet specifically requests action by a 
particular application server. • . -: 

30 Once the packet filter module 74 determines which application server 

should receive the data packet, the packet filter module at step 216 re-writes 
the MAC address and optionally, re-writes, the IP address and TCP/UDP port 
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number in the header of the dataipacket ;to> reflejpt the address of the selected 
application server. Then/ at step^ tfte connection 

table 66 to reflect the new cortneGtion. ^.iT^ p^c^et filte;^ n^odule 74 ; ;then 
returns the modified date packet back* to th^ 72 at step 

5 224 for forwarding to the appropriate networks interface f £7- - : The : packet filter 
module 74 then returns toStep 202 to process the next available data packet. 

If it was determined at step; 2*10 that the received data packet was not a 
new connection with the clientvJhepaGket filter module 74 determines at step 
212 whether a correspohdihg;eniry in, the- connection table 66 exists. If there 

10 is no corresponding entry, a vreset packet is sent for TCP connections or the 
packet is discarded > for UDP- connections at step 208... Conversely, if, the 
connection table 66 has a corresponding eotry ; for the jdata packet, .then, at 
step 220, the packet filter module -74; re-writes the .MAC a and 
optionally re-writes the IP address and TCP/UDP port numbeRtp reflect the 

15 application* server and application that 4s, : already servicing the .connection. 
The packet filter module* 74 then returns the„ modified, d;ata,paqket back to the 
packet engine moclule 72 at step 224 for forwarding -to the appropriate 
' : network interface 37. The packet filter module 74 then Returns to step 202 to 
process the next available ^data packet;;^ ^ , ^ ^ v:- : .. 

20 Conventional dedicated ^switching processors^ inqludea control, module 

and a switching module that- ase. viewed.*ky i;the 'externah r^tv^orks.as a single 
network entity. The control module; ;comm t unicates. ^ module 
through special internal Interfaces that ' add, overhead -,tp>, both- the control 
module and the switching modules, and :is thus undesirabJe. An : advantage of 

25 the toad balancing andTDack'et^switcfjing system to of the present invention is 
that the - control processor :42 and the. switching processors. -44i-44 2 may be 
viewed as entirely, separate logical networking end points even though they 
both reside within a single physical device. Therefore, external clients may 
communicate with applications running on the control processor 42 by 

30 sending data packets through the switching , processors 44^442, which, in 
turn, route the data packets to the control processor. The control processor 
42 reverses the order to send data packets backto the external clients. 
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A first alternative fembodirti^ht of the invention is provided in Fig. 7, 
which illustrates si block diagram i of "ia pseudo^interface between the control 
processor 42 and the sWitchirig processors 44i-44 2 . As discussed above with 
respect fo '^ switching device 10 

5 communicates bn : the client side through an external switch 13 and on the 
badk-end server side through ah internal ' switch 11. More particularly, the 
switching processor 44 1 communicatee with the external switch 13 through the 
network interface 37^ the switching 1 processor 44 2 communicates with the 
external switch 13 through the network interface' 37 2 . Similarly, the switching 

10 processor 44i "communicated with the infernaT Switch ^11 through the network 
interface 37 3 , and 'the switching proc^sbr44 r 2 communicates with the internal 
switch 1 T through the : network 'interface 37 4 P The control processor 42 also 
communicates with 1 the internal switch 1 1 through the network interface 37 0 . 
A virtual IP addresses assigned to 'the network interface 37 0 . When 

15 external devices on the client side of the netWork wish to communicate with 
' the control processor 42, a data packet is transmitted through the external 
' switch 13 to one of the switching processor^ -44' 1 -44 2 , with the IP header 20c 
of the <iiata Ipadket listing the virtual IP address as the destination. The 
switching prodessor 44 then' processes 4 the incoming data packet in the 

20 manner described above with respebt td Figfc. 5 and 6. Specifically, the 
packet filter mbdUlb 74 of ffter sWitchin^ processor re-writes the IP header 20c 
ofihe data piackfettcf nefleSt ^lie^riariP Address of the network interface 37 0 . 
The packet engine "modlilie 72 trieh ; routes the modified data packet to the 
internal switch 11 through a corresponding one of the- network interfaces 37. 

25 1 The internal switch 11 then s^nds the modified data packet to the network 
interface 37 0 which then delivers the data packet to the control processor 42. 
The process is reversed for responses sent by the control- processor 42 back 
to the external device that originated the connection. The control processor 
42 sends a data packet via the network interface 37 0 having the real IP 

30 address through the internal switch 11 to one of the switching processors 44. 
The switching processor 44 re-writes the IP address to the virtual IP address 
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known to the external device: .The modified, data packet is .then, sent out by 
the switching processor^ through^he exte 

A second alternative embodiment of the invention is provided in Fig. 8, 
which, illustrates a bjock. diagram Qf:.a pseudo-interface, between the control 
5 processor 42 and a switching processor . t 44... The control processor 42 actually 
c.- operates at two levels in atime-shar^ manner,, referred ..to. as a user level 
and an operating system level.; l V The v user level .comprises the systems 
accessible to the user,, and ^rnay , include one or more user, application 
programs 51 executing: thereon,, 3uch as an e-maj! .program,, a server 

10 application, and/or an Internet .browser. ir The resource manager .52 and 
master module 54 described abpy.a,v/i.th respect to .Fig.. 4 also execute in the 
user level. The operating, system level,, also known ^s the kernel, provides 
the basic services for th^ ; cpntrpJ r prqcessor .42 as : well as the switching 
processor 44 f such as, activating t the hardware directly or interfacing to 

15 - another software layer thatdrives the hardware. . . 

As shown in Fig. 8, : th^-operating system. 48 further includes a protocol 
module 55, a pseudo-network driver. 57, and a network driver 59. The 
protocol -module 55 serves as a , data : interface for the user application 
programs 51. The protocol rrupdule, 55 converts received data packets Jhat 

20 are directed to one of the u§er application programs 51 from the TCP/JP or 
UDP/IP protocols into, a foTOa[t V} usah jus^r ^pRli^tion programs. 

Specifically, the protocol mpdule .,5.5 sirip^pf^h^ header 
20c, and TCP headen20b ; , leaving the, data portion 20a pf. the data packet. 20 
(see Fig., 9). The data portion 20a. is then provided to the user application 

25 programs 51 . Conversely, the protocol module 55 formats data sent out from 
the user application programs 51. into, data packets in accordance with the 
TCP/IP. or UDP/IP protocols, by adding .the MAC header 20d, IP header 20c, 
- ...and TCP (or UDP) header 20b. . 

. The network drivers 59 provide an interface between the hardware 

30 network interfaces 37 and the software switching processor 44. As illustrated 
in Fig. 8, the control processor 42 does not have a direct connection to the 
network drivers 59. Instead, the pseudo-network driver 57 is configured to 
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' appearto the user application programs 51 as; a hardware network interface. 
'The pseudo-network 3 driver 57 may be provided by a STREAMS mechanism, 
which is a feature of a UNIX-based system that provides a standard way of 
dynamically building and passing nriesisages up and down a message stack. 
5 Ordinarily, messages from Vu^er application are passed "downstream" to the 
' network driver "at the eridof the stack,, and -messages from the network driver 
are passed "upstream" to the user application. In the present invention, the 
pseudo-network driver 57- provides a message stack that is accessed through 
the use of system calls issued by^ the :-user application programs 51 to 
10 communicate with remote devices through- the pseudo-network driver 57. As 
will be further described below, a data packet storage area 68 within the 
shared memory 34 appears to the user application programs 51 as such a 
'remote device. v ' ' 

jhe interface daemon 53 is a 1 program that executes in the background 
15 in the user level of the control processor 42 to communicate with the switching 
processor 44 and the pseudo-network driver 57 to initiate transfers of data 
packets therebetween. As described above with respect to Figs. 5 and 6, the 
switching processors 44 receive iricbming data packets from remote devices 
' through the network interfaces" 37! •'■ At step«:204 of Fig. 6, the packet filter 
20 module 74 reads the MAG f address' ahd'IP, information from the header of a . 
received dsita packet in order-to determine routing of the data packet. If the 
packet switching pVb6ess6p44^ iat step 204. that the intended 

' destination for the data packet : is<an£ of- the: user applications 51 running on 
: the control processor 42, the data packet is written into the data packet 
25 storage : location 68 of the shared memory 34. The switching processor 44 
then signals the interface daemon 53' of- the availability of the data packet. 
The interface daemon 53 moves the received data packet to the pseudo- 
network driver 57. The received data packet is then processed through the 
protocol module 55 as if it were' an incoming data packet received through an 
30 actual network interface. 

To send data packets that originate in one of the user applications 51 
to a remote device, the foregoing process is reversed. More particularly, data 
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packets from the user, applications ,51 ^ajne, pa^e^ to : the pseudo-network 
driver 57, ahd the interface daemcyp 53 mppitqrsthepse^^ driver for 

data packets. Qnce a data packet am ves j; at.the pseudo-network driver from 
the user application^, the. inte/fa^ 
5 -places it in the data packejt storage iopatipp^S of the .shared memory 34. , 
: Then, the interface daemon, ; 53. sjgn.als the swrtching :; proces : sor 44 of the 
availability of the data packeWP;;tb^data packet storage location 68. The 
switching processor 44 then.-: retrieve? - the data; packet, fronp the shared 
memory 34, and routes the data packet tprjone-.of tfre network interfaces. 37 in 

10 the same manner as descrrked^akoye: _As a, .result, remote devices can 
communicate with user applications [51 rynning on ihe control processor 42 
even though the control pto.cessQr. does not have a direct, connection to a 
network interface. The user applications 51 executing on, the control 
processor 42 think they are* ^qpmmunicating ..directly with actual network 

15' interfaces. - :v .-if :-v ■ • . •= , ;?-.! ; v - 

As discussed above, usfer. applied the user 

level, and data input and output is handled jat the operating, $ystem ; level. . The 
operating system communrQates : ;yyith .the, network devices and issues 
interrupts to the network applications at .tha user leyel to indicate the receipt of 

20 data. These conventional;- systems, ar^ inefficient $ince r; processing of the^ 
network applications is stopped>;.ea^ 

involvement of the operating systernrfu^ efficiency of thp user 

applications. It would therefore be d;e^rable : ,ta^ 

direct access to the network interfaces and to f run. the network applications on 
25 a dedicated processor which, is not interrupted. Certain : network .applications 
such as Internet telephony or fax applications would particularly benefit from 
such direct network access. . 

. A third embodiment of the invention is provided at Fig. . 10,. which 
illustrates a block diagram .of a . user-level network interface for applications 
30 running on the switching processor 44. The user-level network interface 
overcomes the inefficiencies of the conventional systems discussed above. In 
. Fig. 10, the switching processor 44 has certain network applications 65 



0O30322A2 I _> 



WO 00/30322 PCJ/US99/27048 

.20 

.^running -thereon, including, the jacket switching functions, described above. 
-The net\wprk applications 1 65 : ^nd the- packet .switching program have direct 
' access to a list of buffers .in the kerr^el memory 63; In an Ethernet network, 
each network interface .-37 has a : list, of buffers associated with it. These 
5 buffers can be used -to transmit data as.well as receive data. A network driver . 
• : 59 on the, .operating system 48 communicates with the network interface 37 in 
the manner deseribed> previously, and : also has access to the buffer list in the 
■ kernel memory 63. : r . .„ ,r ,; , . • .. 

In particuljar, the .descriptors identify the, address of the buffer within the 
10 kernel. memory 63 r the- ,length of the date stored in the buffer, and an 
ownership identification of the buffer^. (Le^., whether the buffer is presently 
"owned", or controlled by the network interface , hardware or the network 
application software). The network interface, 37 circles through the buffer list 
- in the kernel memory 63 to access the -buffers in order to. send or receive data 
15 as .necessary. Similarly; the -network . applications 65 on the switching 
processor. 44. circle through the. list of, buffers to process, the data. If the 
network, interface 37 transmits data from a particular buffer, the network 
. applications 65 reclaim the buffer H and , return it to a free buffer pool. 
, < Qonversely, if the network interface -37 .has just received data and placed the 
20 data in. a particular buffer; the network applications 65 process the data. 

Figsr,; 11 -1i3 ; illustrate by the switching 

processor 4;4 : to initiate t .thevdire^ to the network interfaces, to 

. - send /jata packed to receive data packets from 

•thejietwork interfaces.; As shown in Fig. 11,* the switching processor 44 is 
25 initiated in a process beginning at step 300. Atstep 301, all interrupts to the 
switching processor 44 are disabled so that the switching program and any 
network application programs are run exclusively . on the processor. Any 
interrupts from any device are thereafter delivered to the control processor 42. 
Next, at step 302, the kernel memory 63 that is to be .shared between the 
30; network interfaces 37 and the network applications 65 operating on the 
switching processor 44 is allocated. All the buffers within the kernel memory 
63 are mapped to all of the, network interfaces 37 so that any buffer can be 
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used to transmit or receive data' tKrbudjh any of the network interfaces. Lastly, 
at step 303, the network interfaces^ registers khd buffers-are mapped to the 
network applications 65.- This eriables"the-network applications 65 to directly 
control the network interfaces 37 by" changihg'the^-conterit of the registers and 
5 to perform read/wnte operations frdrnVtaHhet bQffers directly. 1 ' - - 

Once the switching processor- 44 is initiated in this manner, all data 
accesses from/to the netWork'Iriterfaces operate like conventional memory 
read/write operations by the network applications. High efficiency results from 
the fact that the network applications' 65 ahd : the switching prdgram run on a 

10 single thread on a dedicated , -nbh-ihtferruptable^process6r. v'Alsb, there is no 
context switching since the prograhns running oh the 1 switching processor 44 
are isolated as a separate grdu^ thaHs' not available any other processes 
or threads in the multiprocessor system. ■ w 

The process of sending cteta from one of the network applications 65 to 

15 the network interface 37 is illtistrated in Fig. 12, and begins with step 320. At 
step 321, the network application -6^ gets -the next availabte buffer from the ' 
free buffer pool. The free buffer piool rnay be maintained as-aUable within the 
kernel memory 63. The netWork : Application* 65 then writes the* data to be 
transmitted in the form of a <Sata< p^ckiet into ^the identified buffer at step 322, 

20 and changes the "ownership" "bf^ ^the "buffer to the 1 network interface 37 at step 
323. At step 324, tfre' rfetwb?^^ network 
interface 37 that a buffer cohteirt^batS-r4aa^t(^ Be nteW^Itfecli At step 325, 
the network application 65 ; per\6'd\6My checks-io^ee 'if the data has been 
transmitted. Once the data has been trarisrhittedi the network application 65 

25 returns the buffer to the Tree pool at step 326. At step 327, the network 
application 65 returns to perfornling other tasks. 

: The process of receiving data from "the network interface 37 to one of 
the network applications 65 is illustrated in Fig.' 13, and begins with step 340. 
" At step 341, the network application 65 passes^ list of available buffers from 

30 : the free buffer pool to the network interface 37. At step 342, the network 
application 65 checks the status of the network interface 37 to see if data has 
been received. If no data has been received, step 343 causes the program to 
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loop back and repeat step ;lf data has been received by the network 
interface 37, the network application 65 identifies the buffer into which the 
data has been received by ; .checking., the ownership bit at step 344. The 
network application 65 next.ye.rifies.that valid data was received into the buffer 
5 at step^ 345 Vl and if the data is not yalid then the program returns to step 342. 
Conversely,, if the received data , i$. valid v then the network application 65 
processes the data at step 346. Thereafter, the network application 65 
returns the buffer to the free, buffer.ppol at step 347. At step 348, the network 
application,65 returns to performing .other tasks. 

10 Having thus described a preferred embodiment of a computer data 

. packet switching, and : load balancing system using a general-purpose 
symmetrical multiprocessor architecture, it should be apparent to those skilled 
in the art that certain advantages of the, aforementioned system have been 
achieved. It should also be appreciated that various modifications, 

15 , adaptations, and alternative embodiments thereof may be made within the 
scope and spirit -of the. present invention. The. invention is further defined by 
the following claims.. < - , * - 

: '- : •" . j .* ' r . $ ■ , *■ ■ j~ ...j ( > v. ; . 
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. .-n CLAIMS :o v\ ivr ■ 

What is Claimed is: 1 1 " ' ' * ! 

1 / A general-purpose multfproceksdr computer system comprising: 
a plurality 'of network interfaces each adapted to be coupled to 
5 respective external networks 'for receiving ^nd Sending data packets to 

and from said external 1 rietWorks" via' *a particular communication 
protocol; ' ~ • ' -° — r *<■ 

a plurality of proctors including- a first processor providing a 
control processor sind remaining ones of said 1 processors each 
10 providing data packet switbhihg processors, said data packet switching 

processors each being coupled t6 each one of said' plurality of network 
interfaces; and " " v " " : 

a main memo^ ; febup1ecd to each of said : plurality of processors, 
said control processor receiving raw load status data from said "external 
15 networks and genersitihg Ibad distribution configuration data therefrom, 

said load distributioh cohfiguratioh data being stored in said main 
memory for access by said data packet switching-processors, said 
switching processors routing received ones of said data packets to a 
selected one of said external networks in accordance with information 
20 included in a header portion of said data packets and said load 

distribution configuration data. 

2. The computer system of Claim 1, wherein said switching 
processors further provide periodic polling of corresponding ones of said 

25 network interfaces for detecting a received one of said data packets therein. 

3. The computer system of Claim 1, wherein said switching 
processors further re-write said routing information included in said header 
portion of said data packets to reflect said selected one of said external 
networks. 
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4: !i The computer 'System of Claim 1- wherein said main memory 
further comprises a eonhectibh tabled reflecting status of previously received 
: ones of saTd data packets; 1 "'^ ■ ■» * • 

5. ...The.. computer, system.. of.. Claim 4, wherein said switching 
5 processors access said connection table to determine correspondence 

, between a received jDne of said data, packets and said previously received 
ones of said data packets in determining said selected one of said external 
networks., . , 

6. The computer syktenrrof Claihrfi, wherein said" main memory 
10 further comprises a configuration table ^containing said load distribution 

configuration data.' >^ 

7. The computer system of Claim, 1, wherein said communication 
protocol comprises Transmission Control Protocol/Internet Protocol (TCP/IP). 

8;, The computer, system , of Claim 1, wherein said switching 
15 processors further include an engine module:,^aving stored instructions to be 
executed .by -said- switching processors, : said^ i( stored instructions comprising 
the steps of: 

. v polling a first one of. said network interfaces for presence of a , 
Mi . ; ^ receivgcLdata ipapkpt;; c? :o ^jfea pr:;-;^ 
20 if a received data packet is present at said first one of said 

/ / network interfaces, -rpu^ packet to said selected 

... one of said external networks, and : . t . 

if a received one of said data packets is not present at said first 
one of said network interfaces, polling another one of said network 
25 interfaces for presence of a received data packet. 
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9. .. . ■ The computer system , oh CJfiim,^ : 1 c . wherein said switching 
processors . further include ;a Mer.< module rhaving- .stored .^ be 
executed by said switching processors, said stored instructions, comprising 
the steps of: 

5 "* ' reading routing information frbm said header portion of said data 

packet, 

accessing said load distribution "configuration data stored in said 
main memory, r ,t 

selecting said selected one of said external networks 1 based on 
10 said routing informatipn^nd-said load djstrijDgtipn configuration data, 

. modifying saicTdaJa packet, by re-writing ^aid : routing information 
to reflect said selected one of said external networks, and 

sending said modified data packet to one of said plurality of 
network interfaces 'corresponding to said 1 selected bh6 of said external 
15 networks. ! ' r -".-^v. 

10. The computer -systehrv of Claim "9, 1 wherein said stored 
instructions of said filter module l^rtKer comprte^ 

■ reading transport Inform^ said 

data packet, and 

20 accessing connection sfatus : da'ta stotfed in af-Gdlnnection table of 

said main memory reflecting status of previously i% : e£f\^d ones of said 
data packets, ' " * ,l sj^b ~ \ 

wherein, if'iaid transport" infofrtiation- indicated that said data 
packet corresponds to a previously received data packet, then said 
25 selecting step further comprises selecting said selected one of said 

' < external networks based on routing of said previously received data 
packet. ... 
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> 11. • ln . =a > general-purpose multiprocessor computer system 

comprising- a ^:plurality^of. symmetrica]; processors, a main memory shared by 
said plurality of processors, and a plurality of network interfaces each adapted 
to be coupled to respective external networks for receiving data packets 
5 therefrom and sending data packets thereto via a particular communication 
protocol, a method for operating said computer system comprises the steps 

of: ' ' " C " 

configuring one of said plurality of processors as a control 
processor and remaining ones of said processors as data packet 
10 switching processors, said data- : packet switching processors each 

.< - being coupled to each one of sjaid glgrality of network interfaces; 

providing load data regarding load status of said external 
networks; 

""generating load ^distribution Configuration data from said load 
15 data using said control processor arid storing said load distribution 

configuration data in said main memory for access by said data packet 
switching [ ptodessors; and 
* " ' routing received ones- of said data packets using said switching 
processors to a selected one of r said Sxternil networks in accordance 
20 ' * with information included in a- header portion of said data packets and 
5 said Ib^d distribution cbhfigu^^ 

12. The method of Claim 11, further comprising the step of 
periodically polling corresponding ones of said network interfaces by said 

25 switching processors for a received one of said data packets. 

1 3. The method of Claim 1 1 , further comprising the step of re-writing 
said routing information included in said header portion of said data packets 
by said switching processors to reflect said selected one of said external 
networks. 
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14. The method -;of~ Claim: 11 v .furttter comprising . providing a 
connection table in said main memory: reflecting; status ;qf. previously received 
ones of said data packets.- : -iu.e , ; : /\. ! i? . , 

15. The method of. Claim .14, further comprising accessing said 
5 connection table by said switching processors to determine correspondence 

b e t W een a received one of said data packets and said previously received 
ones of said data packets in determining said selected, one of said external 
networks. ; _ 

16. the method* A ' : 6f : ' Claim 11, further comprising providing a 
10 configuration table in said v rtiS^ 

configuration data. ' : ' - : > ' ; ^ - 

1 7. , The method of Claini, 1 1, further comprising the steps of: 

polling a . first x>ne . of said , network interfaces for presence of a 
received data packet, (T ^ n ( . , rv . f . , . !f ... . ir:! , . f v . 
15 if a received data packet is present .gt said first one of said 

network interfaces, rputing. s^iid, received data papket to said selected 
one of said externa] networks, and . ; , s, :r . ;: 

if a received #ne-„p£;§jplci : dat£ packets jS: not present at said first , 
one of said network. i$erf^ce^ network 
20 interfaces for presence of a received data packet. 
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18. The method of Claim 1 1 , further comprising the steps of: 
reading routing information from said header portion of said data 

packet, 

accessing said load distributiqn configuration data stored in said 
5 main memory, 

selecting said selected one' of said external networks based on 
said routing information and said load distribution configuration data, 

modifying said data packet by re-writing said routing information 
to reflects said selected one of said external networks, and 
10 sending said modified data packet to one of said plurality of 

network interfaces corresponding to said selected one of said external 
"networks. ' p; 

19. The method of Claim 18, further comprising the steps of: 
reading transport information from said header portion of said 

15 _ data packet, and j 

accessing connection status data stored in a connection table of 
.said main memory reflecting status of previously received ones of said 
' ■ : data packets, j - < ■ 

wherein, if said trafisporft ihfdrrfiation indicates that said data 
20 packet Icorresponds to a previously received data packet, then said 



30 



4 selecting step further comprises' ^ledting said selected one of said 
externa! networks! based on rolitihg : pf said previously received data 
v packet. ' ° 



25 20. The method of Claim 1 1 , further comprising the ; step of providing 

an active and a back-up portion of said" main memory; wherein newly 
generated load distribution configuration data is stored in said active portion 
while said back-up portion is accessible to said switching processors. 
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